The Size Distortion of Bootstrap Tests

We provide a theoretical framework in which to study the accuracy of bootstrap P values, which may be based on a parametric or nonparametric bootstrap. In the parametric case, the accuracy of a bootstrap test will depend on the shape of what we call the critical value function. We show that, in many circumstances, the error in rejection probability of a bootstrap test will be one whole order of magnitude smaller than that of the corresponding asymptotic test. We also propose a simulation method for estimating this error that requires the calculation of only two test statistics per replication. This research was supported, in part, by grants from the Social Sciences and Humanities Research Council of Canada. Earlier versions were presented at Universidad Carlos III de Madrid, Universidad Complutense de Madrid, Cambridge University, INSEE-CREST (Paris), CORE (Louvain-la-Neuve), the Tinbergen Institute (Amsterdam), the University of Geneva, the European University Institute (Florence), the ESRC Econometrics Conference (Bristol), the 1996 Berkeley Symposium on the Bootstrap, the Canadian Econometric Study Group, Queen’s University, Laval University, and the University of Texas (Austin). We are grateful to many seminar participants and to three anonymous referees for comments. We are especially grateful to Joel Horowitz, not only for comments, but also for his probing questions that led us to clarify the paper. The first draft of the paper was written while the second author was visiting GREQAM.